Development and validation of a machine-learning prediction model to improve abdominal aortic aneurysm screening.

OBJECTIVE: Despite recommendations by the United States Preventive Services Task Force and the Society for Vascular Surgery, adoption of screening for abdominal aortic aneurysms (AAAs) remains low. One challenge is the low prevalence of AAAs in the unscreened population, and therefore a low detection rate for AAA screenings. We sought to use machine learning to identify factors associated with the presence of AAAs and create a model to identify individuals at highest risk for AAAs, with the aim of increasing the detection rate of AAA screenings. METHODS: A machine-learning model was trained using longitudinal medical records containing lab results, medications, and other data from our institutional database. A retrospective cohort study was performed identifying current or past smoking in patients aged 65 to 75 years and stratifying the patients by sex and smoking status as well as determining which patients had a confirmed diagnosis of AAA. The model was then adjusted to maximize fairness between sexes without significantly reducing precision and validated using six-fold cross validation. RESULTS: Validation of the algorithm on the single-center institutional data utilized 18,660 selected patients over 2 years and identified 314 AAAs. There were 41 factors identified in the medical record included in the machine-learning algorithm, with several factors never having been previously identified to be associated with AAAs. With an estimated 100 screening ultrasounds completed monthly, detection of AAAs is increased with a lift of 200% using the algorithm as compared with screening based on guidelines. The increased detection of AAAs in the model-selected individuals is statistically significant across all cutoff points. CONCLUSIONS: By utilizing a machine-learning model, we created a novel algorithm to detect patients who are at high risk for AAAs. By selecting individuals at greatest risk for targeted screening, this algorithm resulted in a 200% lift in the detection of AAAs when compared with standard screening guidelines. Using machine learning, we also identified several new factors associated with the presence of AAAs. This automated process has been integrated into our current workflows to improve screening rates and yield of high-risk individuals for AAAs.

Who needs what (features) when? Personalizing engagement with data-driven self-management to improve health equity.

OBJECTIVES: To examine the feasibility of promoting engagement with data-driven self-management of health among individuals from minoritized medically underserved communities by tailoring the design of self-management interventions to individuals' type of motivation and regulation in accordance with the Self-Determination Theory. METHODS: Fifty-three individuals with type 2 diabetes from an impoverished minority community were randomly assigned to four different versions of an mHealth app for data-driven self-management with the focus on nutrition, Platano; each version was tailored to a specific type of motivation and regulation within the SDT self-determination continuum. These versions included financial rewards (external regulation), feedback from expert registered dietitians (RDF, introjected regulation), self-assessment of attainment of one's nutritional goals (SA, identified regulation), and personalized meal-time nutrition decision support with post-meal blood glucose forecasts (FORC, integrated regulation). We used qualitative interviews to examine interaction between participants' experiences with the app and their motivation type (internal-external). RESULTS: As hypothesized, we found a clear interaction between the type of motivation and Platano features that users responded to and benefited from. For example, those with more internal motivation reported more positive experience with SA and FORC than those with more external motivation. However, we also found that Platano features that aimed to specifically address the needs of individuals with external regulation did not create the desired experience. We attribute this to a mismatch in emphasis on informational versus emotional support, particularly evident in RDF. In addition, we found that for participants recruited from an economically disadvantaged community, internal factors, such as motivation and regulation, interacted with external factors, most notably with limited health literacy and limited access to resources. CONCLUSIONS: The study suggests feasibility of using SDT to tailor design of mHealth interventions for promoting data-driven self-management to individuals' motivation and regulation. However, further research is needed to better align design solutions with different levels of self-determination continuum, to incorporate stronger emphasis on emotional support for individuals with external regulation, and to address unique needs and challenges of underserved communities, with particular attention to limited health literacy and access to resources.

Examining AI Methods for Micro-Coaching Dialogs.

Conversational interaction, for example through chatbots, is well-suited to enable automated health coaching tools to support self-management and prevention of chronic diseases. However, chatbots in health are predominantly scripted or rule-based, which can result in a stagnant and repetitive user experience in contrast with more dynamic, data-driven chatbots in other domains. Consequently, little is known about the tradeoffs of pursuing data-driven approaches for health chatbots. We examined multiple artificial intelligence (AI) approaches to enable micro-coaching dialogs in nutrition - brief coaching conversations related to specific meals, to support achievement of nutrition goals - and compared, reinforcement learning (RL), rule-based, and scripted approaches for dialog management. While the data-driven RL chatbot succeeded in shorter, more efficient dialogs, surprisingly the simplest, scripted chatbot was rated as higher quality, despite not fulfilling its task as consistently. These results highlight tensions between scripted and more complex, data-driven approaches for chatbots in health.

Design and Evaluation Challenges of Conversational Agents in Health Care and Well-being: Selective Review Study.

BACKGROUND: Health care and well-being are 2 main interconnected application areas of conversational agents (CAs). There is a significant increase in research, development, and commercial implementations in this area. In parallel to the increasing interest, new challenges in designing and evaluating CAs have emerged. OBJECTIVE: This study aims to identify key design, development, and evaluation challenges of CAs in health care and well-being research. The focus is on the very recent projects with their emerging challenges. METHODS: A review study was conducted with 17 invited studies, most of which were presented at the ACM (Association for Computing Machinery) CHI 2020 conference workshop on CAs for health and well-being. Eligibility criteria required the studies to involve a CA applied to a health or well-being project (ongoing or recently finished). The participating studies were asked to report on their projects' design and evaluation challenges. We used thematic analysis to review the studies. RESULTS: The findings include a range of topics from primary care to caring for older adults to health coaching. We identified 4 major themes: (1) Domain Information and Integration, (2) User-System Interaction and Partnership, (3) Evaluation, and (4) Conversational Competence. CONCLUSIONS: CAs proved their worth during the pandemic as health screening tools, and are expected to stay to further support various health care domains, especially personal health care. Growth in investment in CAs also shows the value as a personal assistant. Our study shows that while some challenges are shared with other CA application areas, safety and privacy remain the major challenges in the health care and well-being domains. An increased level of collaboration across different institutions and entities may be a promising direction to address some of the major challenges that otherwise would be too complex to be addressed by the projects with their limited scope and budget.

A deep database of medical abbreviations and acronyms for natural language processing.

The recognition, disambiguation, and expansion of medical abbreviations and acronyms is of upmost importance to prevent medically-dangerous misinterpretation in natural language processing. To support recognition, disambiguation, and expansion, we present the Medical Abbreviation and Acronym Meta-Inventory, a deep database of medical abbreviations. A systematic harmonization of eight source inventories across multiple healthcare specialties and settings identified 104,057 abbreviations with 170,426 corresponding senses. Automated cross-mapping of synonymous records using state-of-the-art machine learning reduced redundancy, which simplifies future application. Additional features include semi-automated quality control to remove errors. The Meta-Inventory demonstrated high completeness or coverage of abbreviations and senses in new clinical text, a substantial improvement over the next largest repository (6-14% increase in abbreviation coverage; 28-52% increase in sense coverage). To our knowledge, the Meta-Inventory is the most complete compilation of medical abbreviations and acronyms in American English to-date. The multiple sources and high coverage support application in varied specialties and settings. This allows for cross-institutional natural language processing, which previous inventories did not support. The Meta-Inventory is available at https://bit.ly/github-clinical-abbreviations .

Automated vs. Human Health Coaching: Exploring Participant and Practitioner Experiences.

Health coaching can be an effective intervention to support self-management of chronic conditions like diabetes, but there are not enough coaching practitioners to reach the growing population in need of support. Conversational technology, like chatbots, presents an opportunity to extend health coaching support to broader and more diverse populations. However, some have suggested that the human element is essential to health coaching and cannot be replicated with technology. In this research, we examine automated health coaching using a theory-grounded, wizard-of-oz chatbot, in comparison with text-based virtual coaching from human practitioners who start with the same protocol as the chatbot but have the freedom to embellish and adjust as needed. We found that even a scripted chatbot can create a coach-like experience for participants. While human coaches displayed advantages expressing empathy and using probing questions to tailor their support, they also encountered tremendous barriers and frustrations adapting to text-based virtual coaching. The chatbot coach had advantages in being persistent, as well as more consistently giving choices and options to foster client autonomy. We discuss implications for the design of virtual health coaching interventions.

From Reflection to Action: Combining Machine Learning with Expert Knowledge for Nutrition Goal Recommendations.

Self-tracking can help personalize self-management interventions for chronic conditions like type 2 diabetes (T2D), but reflecting on personal data requires motivation and literacy. Machine learning (ML) methods can identify patterns, but a key challenge is making actionable suggestions based on personal health data. We introduce GlucoGoalie, which combines ML with an expert system to translate ML output into personalized nutrition goal suggestions for individuals with T2D. In a controlled experiment, participants with T2D found that goal suggestions were understandable and actionable. A 4-week in-the-wild deployment study showed that receiving goal suggestions augmented participants' self-discovery, choosing goals highlighted the multifaceted nature of personal preferences, and the experience of following goals demonstrated the importance of feedback and context. However, we identified tensions between abstract goals and concrete eating experiences and found static text too ambiguous for complex concepts. We discuss implications for ML-based interventions and the need for systems that offer more interactivity, feedback, and negotiation.

Enabling personalized decision support with patient-generated data and attributable components.

Decision-making related to health is complex. Machine learning (ML) and patient generated data can identify patterns and insights at the individual level, where human cognition falls short, but not all ML-generated information is of equal utility for making health-related decisions. We develop and apply attributable components analysis (ACA), a method inspired by optimal transport theory, to type 2 diabetes self-monitoring data to identify patterns of association between nutrition and blood glucose control. In comparison with linear regression, we found that ACA offers a number of characteristics that make it promising for use in decision support applications. For example, ACA was able to identify non-linear relationships, was more robust to outliers, and offered broader and more expressive uncertainty estimates. In addition, our results highlight a tradeoff between model accuracy and interpretability, and we discuss implications for ML-driven decision support systems.

Adapting the stage-based model of personal informatics for low-resource communities in the context of type 2 diabetes.

Growing availability of self-monitoring technologies creates new opportunities for collection of personal health data and their use in personalized health informatics interventions. However, much of the previous empirical research and existing theories of individuals' engagement with personal data focused on early adopters and data enthusiasts. Less is understood regarding ways individuals from medically underserved low-income communities who live with chronic diseases engage with self-monitoring in health. In this research, we adapted a widely used theoretical framework, the stage-based model of personal informatics, to the unique attitudes, needs, and constraints of low-income communities. We conducted a qualitative study of attitudes and perceptions regarding tracking and planning in health and other contexts (e.g., finances) among low-income adults living with type 2 diabetes. This study showed distinct differences in participants' attitudes and behaviors around tracking and planning, as well as wide variability in their sense of being in charge of different areas of one's life. Ultimately, we found a strong connection between these two: perceptions of being in charge seems to be strongly connected to an individual's proactive or reactive tracking and planning in that area. Whereas individuals with a greater sense of being in charge of their health were more proactive, meaning they were likely to engage with all the stages of personal informatics model on their own, those with less of a sense of being in charge were more likely to be reactive-relying on their healthcare providers for several critical stages of self-monitoring (deciding what data to collect, integrating data from multiple sources, reflecting over patterns in collected data, and arriving at conclusions and implications for action). Perhaps as a result, these individuals were less likely to experience increases in self-awareness and self-knowledge, common motivating factors to engaging in self-monitoring in the future. We argue that adapting this framework in a way that highlights gaps in individuals' engagement has a number of important implications for future research in biomedical informatics and for the design of new interventions that promote engagement with self-monitoring, and that are robust in light of fragmented engagement.

A method for harmonization of clinical abbreviation and acronym sense inventories.

BACKGROUND: Previous research has developed methods to construct acronym sense inventories from a single institutional corpus. Although beneficial, a sense inventory constructed from a single institutional corpus is not generalizable, because acronyms from different geographic regions and medical specialties vary greatly. OBJECTIVE: Develop an automated method to harmonize sense inventories from different regions and specialties towards the development of a comprehensive inventory. METHODS: The method involves integrating multiple source sense inventories into one centralized inventory and cross-mapping redundant entries to establish synonymy. To evaluate our method, we integrated 8 well-known source inventories into one comprehensive inventory (or metathesaurus). For both the metathesaurus and its sources, we evaluated the coverage of acronyms and their senses on a corpus of 1 million clinical notes. The corpus came from a different institution, region, and specialty than the source inventories. RESULTS: In the evaluation using clinical notes, the metathesaurus demonstrated an acronym (short form) micro-coverage of 94.3%, representing a substantial increase over the two next largest source inventories, the UMLS LRABR (74.8%) and ADAM (68.0%). The metathesaurus demonstrated a sense (long form) micro-coverage of 99.6%, again a substantial increase compared to the UMLS LRABR (82.5%) and ADAM (55.4%). CONCLUSIONS: Given the high coverage, harmonizing acronym sense inventories is a promising methodology to improve their comprehensiveness. Our method is automated, leverages the extensive resources already devoted to developing institution-specific inventories in the United States, and may help generalize sense inventories to institutions who lack the resources to develop them. Future work should address quality issues in source inventories and explore additional approaches to establishing synonymy.

Leveraging Patient-Reported Outcomes Using Data Visualization.

BACKGROUND: Health care organizations increasingly use patient-reported outcomes (PROs) to capture patients' health status. Although federal policy mandates PRO collection, the challenge remains to better engage patients in PRO surveys, and ensure patients comprehend the surveys and their results. OBJECTIVE: This article identifies the design requirements for an interface that assists patients with PRO survey completion and interpretation, and then builds and evaluates the interface. METHODS: We employed a user-centered design process that consisted of three stages. First, we conducted qualitative interviews and surveys with 13 patients and 11 health care providers to understand their perceptions of the value and challenges associated with the use of PRO measures. Second, we used the results to identify design requirements for an interface that collects PROs, and designed the interface. Third, we conducted usability testing with 12 additional patients in a hospital setting. RESULTS: In interviews, patients and providers reported that PRO surveys help patients to reflect on their symptoms, potentially identifying new opportunities for improved care. However, 6 out of 13 patients reported significant difficultly in understanding PRO survey questions, answer choices and results. Therefore, we identified aiding comprehension as a key design requirement, and incorporated visualizations into our interface design to aid comprehension. In usability testing, patients found the interface highly usable. CONCLUSION: Future interfaces designed to collect PROs may benefit from employing strategies such as visualization to aid comprehension and engage patients with surveys.

Visualizing the Patient-Reported Outcomes Measurement Information System (PROMIS) Measures for Clinicians and Patients.

Congruent with the nationwide movement toward patient-centered healthcare, an increasing number of organizations collect and assess patient-reported outcomes (PROs). The standardized NIH PROMIS measures represent one of the most widely used PRO questionnaires, but organizations still face challenges with conveying PROMIS outcomes to clinicians in clinically relevant ways. Our proposed solution, the ProVis application, uses visualizations to engage heart failure patients with PROMIS questionnaires in the waiting room, and conveys PROMIS data to clinicians through longitudinal visualizations in iNYP, our institution's electronic health record (EHR) interface. Here, we discuss the design and development of ProVis, the alternative strategies we considered, the strengths and weaknesses of ProVis, and our future dissemination and evaluation plans.